Fluorescence in situ hybridization (FISH) dot counting is the process of enumerating chromosomal abnormalities in interphase cell nuclei. This process is widely used in many areas of biomedical research, especially in cancer diagnosis and treatment decisions. For example, breast cancer is the most common cancer for women worldwide. Human Epidermal Growth Factor Receptor 2 (HER2) gene copy number is important for breast cancer patients when determining eligibility for adjuvant targeted therapy and/or chemotherapy medications. 1 FISH testing can be used to see if the cells have extra copies of the HER2 gene: a cell having more copies of the HER2 gene is known to have more HER2 receptors, which receive signals that stimulate the growth of breast cancer cells. Using this method, target parts in the cells become visible as colored dots under a fluorescent microscope. The HER2 probes are visible as orange stained spots while probes for the Chromosome 17 centromere (CEP17), the chromosome on which the gene HER2 is located, are visible as green spots. This color-dots visualization method makes it possible to estimate the HER2 gene copy number by counting the dots in the cells. Often, dot counting needs to be approximate and therefore, performed manually. However, in addition to being subjective, manual counting of FISH dots is time-consuming and tiring.
Several researchers have proposed algorithms for automatic counting of FISH dots. For instance, a method based on mathematical morphology and inverse multi-fractal analysis has been used as well as a method based on top-hat filtering and 3-D region growing. Another approach used top-hat filtering followed by template matching to separate real signals from noise. In spite of the use of these algorithms, automatic FISH dot detection is still challenging, especially in the presence of background artifacts (FIG. 1A), dense clusters of cell nuclei and FISH dots (FIG. 1B) and/or low FISH dots contrast (see FIG. 1C). Therefore, there is a persistent need to develop automatic detection and counting algorithms that advance the state of the art and that deal with the FISH dot detection and counting challenges.